At the Edge of Chaos: Real-time Computations and Self-Organized Criticality in Recurrent Neural Networks
نویسندگان
چکیده
In this paper we analyze the relationship between the computational capabilities of randomly connected networks of threshold gates in the timeseries domain and their dynamical properties. In particular we propose a complexity measure which we find to assume its highest values near the edge of chaos, i.e. the transition from ordered to chaotic dynamics. Furthermore we show that the proposed complexity measure predicts the computational capabilities very well: only near the edge of chaos are such networks able to perform complex computations on time series. Additionally a simple synaptic scaling rule for self-organized criticality is presented and analyzed.
منابع مشابه
Neural avalanches at the edge-of-chaos?
Does the brain operate at criticality, to optimize neural computation? Literature uses different fingerprints of criticality in neural networks, leaving the relationship between them mostly unclear. Here, we compare two specific signatures of criticality, and ask whether they refer to observables at the same critical point, or to two differing phase transitions. Using a recurrent spiking neural...
متن کاملMemory Capacity of Input-Driven Echo State Networks at the Edge of Chaos
Reservoir computing provides a promising approach to efficient training of recurrent neural networks, by exploiting the computational properties of the reservoir structure. Various approaches, ranging from suitable initialization to reservoir optimization by training have been proposed. In this paper we take a closer look at short-term memory capacity, introduced by Jaeger in case of echo state...
متن کاملIs cortical criticality unique?
There are indications that for optimizing neural computation, neu-ral networks-including the brain-operate at criticality. Previous approaches have, however, used diverse fingerprints of criticality, leaving open the question whether they refer to a unique critical point or whether there could be several. Using a recurrent spiking neural network as the model, we demonstrate that avalanche criti...
متن کاملRegulation toward Self-organized Criticality in a Recurrent Spiking Neural Reservoir
The goal of the reservoir is to favor rich internal dynamics and interactions with the input signals, as to achieve the required fading memory and kernel properties [1]. There is however no general agreement yet on how the connectivity matrix of the reservoir should be initialized to yield both stability and performance. It is known that edge-of-chaos regime in recurrent neural network leads to...
متن کاملSelf-organized criticality in neural network models
Information processing by a network of dynamical elements is a delicate matter: Avalanches of activity can die out if the network is not connected enough or if the elements are not sensitive enough; on the other hand, activity avalanches can grow and spread over the entire network and override information processing as observed in epilepsy. Therefore, it has long been argued that neural network...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2004